-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use single ES document type #12794
Use single ES document type #12794
Conversation
Signed-off-by: Tyler Smalley <[email protected]>
src/ui/ui_mappings.js
Outdated
@@ -19,7 +19,14 @@ class MappingsCollection { | |||
} | |||
|
|||
getCombined = () => { | |||
return this._currentMappings; | |||
return { | |||
'_default_': { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This shouldn't be necessary right? We can just add this to the doc
type?
Jenkins, test it. |
Signed-off-by: Tyler Smalley <[email protected]>
Signed-off-by: Tyler Smalley <[email protected]>
id, | ||
type, | ||
// handles documents with an _id of `${type}:${id}` | ||
const docResponse = await this._withKibanaIndex('search', { body: createIdQuery({ type, id }) }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When a document is created in 5.x with multi-types, we create the document with the id provided.
When we re-index, we are pre-pending the type to the id due to uniqueness only being guaranteed per-type.
Currently, we are utilizing Elastisearch to provide the unique ID's and if we attempt to create a document with a specific ID, then we continue to prepend it.
I believe we should change using uuid.v1()
or similar and auto-generate the ids ourselves. This would allow us to, beginning in 6.0, to do lookups/updates/gets by ID.
The change here fixes the issue and I am working to implement and test what I mentioned above.
Signed-off-by: Tyler Smalley <[email protected]>
Here is a PR I would like to pull into this branch: tylersmalley#2 @spalger @epixa, I added you to my fork so you're able to merge. |
When creating an id, we will always prefix the type to either an id which was provided or one we generate. This alows us to simplify all actions for managing documents. Signed-off-by: Tyler Smalley <[email protected]>
Signed-off-by: Tyler Smalley <[email protected]>
Remove entire compatibility layer and updates id generation
Signed-off-by: Tyler Smalley <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, but we probably want to extend the health check to go red when the index is using v5-format before merging
My little testing with dashboards and dashboard only mode, looks good to me, no issues! (Minus the index id thing showing up in the prefilled list of indexes, but looks like that might have been mentioned already above?) |
@spalger's changes LGTM |
@tylersmalley I think that's a good idea. |
We need to backport the necessary pieces as well. The longer they sit unbackported, the more likely we'll miss them and cause some really unfortunate regressions with the data, which are the hardest regressions to fix. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
jenkins, test it. It failed on the view edit more test, #12014 |
Actually, failure seems legit, edit: @tylersmalley Looks like we made the wrong call, filtering the fields caused |
Green! |
MERGED! cc: @elastic/kibana-operations Working on a backport for the 5.x specific changes contained in this PR. |
Partial backport containing 5.x specific changes
Partial backport to 5.x: cf5a044 |
@tylersmalley can we get a Release Note: summary of the changes |
Technical Summary:
ensureTypesExist
part of the health check is nowpatchKibanaIndex
, it no longer creates types and applies all patches in a single requestSavedObjectClient
#find()
support for filtering on specific fieldscreateFindQuery()
into pieces and testing each individuallynormalize_es_doc
as the abstractions was subtly different for each method and seemed better implemented inlinefields
andsearchFields
must be arrays after the API.Joi.array().single()
server/mappings
module addedui/ui_mappings
IndexMappings
onkbnServer
that should not be used by code outside of the uiExportsuiExports
typemappings
in a pluginserver.getKibanaIndexMappingsDsl()
, which was previously exposed viakibana.uiExports.mappings.getCombined()
mappings
object and theserver
now just pass around the serverkbnServer.mappings
which is the instance ofIndexMappings
maintained by the kibana server and extended by theUiExports
class usingkbnServer.mappings.addRootProperties()
SavedObject
"types" are now disambiguated from es types by calling them the "root properties" of the "root type" of the "kibana index mappings dsl"SavedObjectLoader
now sends*
(notundefined*
) when search string isundefined
indexPattern.id
toindexPattern.title
, where appropriatemapBreadcrumbs()
argument to routes that will allow them to set the display value for the index pattern id in the breadcrumb as the title of the index patternasync
function that snuck into angular codekibanaServer.uiSettings
to useSavedObjectClient
kibanaIndex
service that fetches a copy of thekibanaIndex
mapping and exposes the name of the kibana index to tests, should they need itretry
is now in the common services, along withkibanaServer
andkibanaIndex
Release Note:
Starting in Elasticsearch 6.0, you are no longer able to create new indices with multiple index types. To accomplish the same effect as index types, a new type field was added to identify the document and the value mapped under the index to allow for strict mapping.
Example
is now